Image Processing Method and Apparatus

ABSTRACT

An image processing method and apparatus are provided. The image processing method includes collecting at least two exposure frames with different brightness in a same scene during different exposure time; combining, for each exposure frame, a raw data unit arranged repeatedly in the exposure frame to obtain first brightness data after the combining; acquiring a correction parameter of all exposure frames according to all first brightness data and performing weighting processing on all the exposure frames by using the correction parameter to obtain a high dynamic range (HDR) image of corrected raw data. The foregoing method can resolve a problem in the prior art that colors, brightness, and contrast of an image obtained from Raw data are severely distorted.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/CN2013/073824, filed on Apr. 7, 2013, which claims priority toChinese Patent Application No. 201210351891.6, filed on Sep. 20, 2012,both of which are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

Embodiments of the present invention relate to communicationstechnologies, and in particular, to an image processing method andapparatus.

BACKGROUND

Research on a high dynamic range (HDR) imaging based on multi-frameexposure combination has been extremely mature. For example, in aconsumer market, an iPhone® is integrated with an HDR photographingfunction. Android® reference platforms of some chips also have arelatively simple HDR imaging function.

Currently, both surveillance cameras for a professional application andmobile phone cameras in the consumer market use a charge-coupled device(CCD) or complementary metal-oxide-semiconductor (CMOS) image sensor tocapture an optical signal in a scene and transform the optical signalinto raw data of a digital signal (that is, Raw data). After beingprocessed by a series of digital signal processing technologies (e.g.,Image Signal Processing (ISP)), the raw signal is transformed into afinal video signal that is suitable for human eyes (generally in redgreen blue (RGB) format or luminance and chrominance (YUV) format).

The Raw data is single-channel data, which is generally formed bymultiple adjacent pixels that are arranged repeatedly and are indifferent colors, and each pixel location has only one value. However,data after color interpolation processing in the ISP increases tothree-channel data, that is, each pixel location has three values.

Most HDR processing technologies are applied to three-channel data afterthe ISP processing, for example, Microsoft® (Special Interest Group onGraphics (SIGGRAPH) 2003 HDR video) proposes a post processing methodfor correcting multiple exposure frames. First, a camera response curveis calibrated by using the multiple exposure frames; and then an HDRimage is combined according to the camera response curve; finally, adynamic range is compressed by using a tone mapping method to improvelocal contrast.

Disadvantages of the foregoing method are as follows. Triple amount ofraw data needs to be processed, and further, such methods as local tonemapping or contrast enhancement need to be used to improve the localcontrast. High complexity is involved. In addition, a gamma correctionin the ISP processing greatly affects brightness of an image. Suchmethod depends on the camera response curve and the tone mapping, and ifthe method is simply applied to the Raw data for direct processing, itmay cause that colors and contrast of the image are seriously distorted.

SUMMARY

In view of the foregoing, for the disadvantages in the prior art, thepresent invention provides an image processing method and apparatus toresolve a problem in the prior art that colors, brightness, and contrastof an image obtained from Raw data are severely distorted.

According to one aspect, an embodiment of the present invention providesan image processing method, including collecting at least two exposureframes with different brightness in a same scene during differentexposure time; combining, for each exposure frame, a raw data unitarranged repeatedly in the exposure frame to obtain first brightnessdata after the combining; and acquiring a correction parameter of allexposure frames according to all first brightness data and performingweighting processing on all the exposure frames by using the correctionparameter to obtain an HDR image of corrected raw data.

According to another aspect, an embodiment of the present inventionfurther provides an image processing apparatus, including a processorconfigured to collect at least two exposure frames with differentbrightness in a same scene during different exposure time; combine, foreach exposure frame, a raw data unit arranged repeatedly in the exposureframe to obtain first brightness data after the combining; and acquire acorrection parameter of all exposure frames according to all firstbrightness data and perform weighting processing on all the exposureframes by using the correction parameter to obtain an HDR image ofcorrected raw data; and a memory configured to store at least twoexposure frames with different brightness that are collected by theprocessor.

It may be known from the foregoing technical solution that according toan image processing method and apparatus in embodiments of the presentinvention, first brightness data of each exposure frame is acquired, andfurther a correction parameter of all exposure frames is acquiredaccording to the first brightness data, so as to perform weightingprocessing on the exposure frame by using the correction parameter toobtain an HDR image of corrected Raw data, thereby resolving a problemin the prior art that colors, brightness, and contrast of an imageobtained from Raw data are severely distorted.

BRIEF DESCRIPTION OF THE DRAWINGS

To describe the technical solutions in the embodiments of the presentinvention more clearly, the following briefly introduces theaccompanying drawings required for describing the embodiments. Theaccompanying drawings in the following description show some embodimentsof the present invention, and a person of ordinary skill in the art maystill derive other drawings from these accompanying drawings withoutcreative efforts.

FIG. 1 is a schematic diagram of an arrangement structure of a Raw dataunit according to an embodiment of the present invention;

FIG. 2 is a schematic flowchart of an image processing method accordingto an embodiment of the present invention;

FIG. 3 is a schematic flowchart of an image processing method accordingto an embodiment of the present invention;

FIG. 4 is a schematic flowchart of an image processing method accordingto an embodiment of the present invention;

FIG. 5A is a schematic diagram of a brightness statistical histogramaccording to an embodiment of the present invention;

FIG. 5B is a schematic diagram of a brightness cumulative histogramaccording to an embodiment of the present invention;

FIG. 5C is a schematic diagram of a brightness mapping functionaccording to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a comparison between an acquired HDRimage and an existing HDR image according to an embodiment of thepresent invention; and

FIG. 7A and FIG. 7B are schematic structural diagrams of an imageprocessing apparatus according to an embodiment of the presentinvention.

DETAILED DESCRIPTION

To make the objectives, technical solutions, and advantages of thepresent invention clearer, the following clearly describes the technicalsolutions of the present invention with reference to the accompanyingdrawings in the embodiments of the present invention. The describedembodiments are a part of the embodiments of the present invention.Based on the embodiments of the present invention, a person of ordinaryskill in the art can obtain other embodiments which can solve thetechnical problem of the present invention and implement the technicaleffect of the present invention by equivalently altering a part of orall the technical features even without creative efforts. Theembodiments obtained through alteration do not depart from the scopedisclosed in the present invention.

A method for combining HDR image based on Raw data is further proposedin the industry. In this method, an exposure time ratio of multipleexposure frames is used as a pixel value of each frame image fornormalization; then an HDR image is obtained by performing a weightedcombination; and a tone mapping method is used to obtain a low dynamicrange (LDR) image suitable for displaying.

However, in the foregoing method, calibration is required in a processof normalizing pixels of different exposure frames by using exposuretime, and the exposure time and the number of gains cannot be the sameas a multiple of pixel brightness. Particularly, the foregoing methodneeds to use many row stores and frame stores, thereby resulting inlarge storage space and a complex calculation process.

Related raw data in the embodiments of the present invention issingle-channel data which is generally formed by multiple adjacentpixels in different colors. As shown in FIG. 1, FIG. 1 shows a schematicdiagram of an arrangement structure of raw data, that is, raw data inBayer format, which is arranged repeatedly in a green red blue green(GRBG) pattern formed by pixels of three colors, red (R), green (G), andblue (B), and each pixel location has only one value.

In the embodiments of the present invention, raw data frames ofdifferent exposure time are mainly combined into an HDR raw data frame.Further, the foregoing process of combining HDR raw may be compatiblewith an ISP processing technology in the prior art. In addition, adynamic range of a final video signal can be effectively expandedwithout changing original colors and local contrast, and storage costsand the amount of calculation of data can be effectively reduced. In thepresent invention, only single-channel data needs to be processed, withno need to calibrate a camera response curve in advance, therebyeffectively reducing the storage costs and the amount of calculation ofdata.

The raw data frame mentioned in this embodiment of the present inventionrefers to an image frame formed by raw data.

With reference to FIG. 1 and FIG. 2, FIG. 1 shows a schematic diagram ofan arrangement structure of a raw data unit according to an embodimentof the present invention, and FIG. 2 shows a schematic flowchart of animage processing method according to an embodiment of the presentinvention. The image processing method in this embodiment is describedas follows.

201. Collect at least two exposure frames with different brightness in asame scene during different exposure time.

In an actual application, data input of different exposures, which maybe two exposures or may be three or more exposures, is required in HDRprocessing. Generally, an optimal exposure frame is referred to as areference exposure frame, and other exposure frames are referred to asextended exposure frames.

In a specific application scenario, multiple exposure frames withdifferent brightness during different exposure time may be collected. Inthis case, the optimal exposure frame is selected as the referenceexposure frame, and the other exposure frames are used as the extendedexposure frames.

202. Combine, for each exposure frame, a raw data unit arrangedrepeatedly in the exposure frame to obtain first brightness data afterthe combining.

For example, the raw data unit in this step may be a data unit arrangedin a blue green green red (BGGR) pattern, a data unit arranged in a GRBGpattern, or a data unit arranged in a green red blue white (GRBW)pattern. This embodiment of the present invention is also applicable toraw data in other patterns.

As shown in FIG. 1, FIG. 1 shows the data unit arranged in the GRBGpattern. That is, in a GRBG Bayer pattern, single-channel data at fouradjacent GRBG pixel locations forms one raw data unit.

Raw data of different image sensor manufacturers is arranged indifferent patterns. However, for all patterns, four or more adjacentpixels form basic units that can be arranged repeatedly. Therefore, acommon Bayer pattern arranged in GRBG is used as an example in thisembodiment of the present invention. After a simple extension, raw datain other patterns, such as a BGGR Bayer pattern and a GRBW pattern, mayalso be used in this embodiment of the present invention.

203. Acquire a correction parameter of all exposure frames according toall first brightness data and perform weighting processing on all theexposure frames by using the correction parameter to obtain an HDR imageof corrected raw data.

It may be understood that the foregoing raw data is single-channel data,and further the finally acquired HDR image of the raw data is also asingle-channel data image.

It may be known from the foregoing embodiment that the raw data unit isa basic unit in this embodiment. First, raw basic units of differentexposure frames are preprocessed, for example, are combined, and thefirst brightness data that is suitable for human eyes is output; andthen, the correction parameter required in the HDR image of thecorrected raw data is calculated, and each piece of raw data in the rawdata unit is combined, so that it is implemented that an existingbrightness post processing effect is combined into the output HDR imageof the raw data.

In addition, in an exemplary embodiment, the acquiring a correctionparameter of all exposure frames according to all first brightness datain the foregoing step 203 may include the following step 2031 to step2033, which are shown in FIG. 3.

2031. Acquire a weighting coefficient of each exposure frame accordingto all the first brightness data.

2032. Perform brightness processing on each exposure frame to obtainsecond brightness data after the processing.

Optionally, a brightness histogram equalization method may be used toperform the brightness processing on each exposure frame to obtain thesecond brightness data after the processing.

2033. Acquire, according to the weighting coefficient, the secondbrightness data, and the first brightness data, a correction factorcorresponding to all the exposure frames, and obtain a correctionparameter that includes the weighting coefficient and the correctionfactor.

Optionally, in another application scenario, the performing weightingprocessing on all the exposure frames by using the correction parameterto obtain an HDR image of corrected raw data in the foregoing step 203includes the following step 2034 not shown in the figure.

2034. Acquire a weighting coefficient, of each pixel in each exposureframe, corresponding to the exposure frame, and acquire a productobtained by multiplying the correction factor by each pixel in eachexposure frame; and summate products corresponding to pixels atcorresponding locations in all exposure frames, and use an image formedby all pixels obtained after the summating as the HDR image of thecorrected raw data.

It may be known from the foregoing embodiment that according to an imageprocessing method in this embodiment, first brightness data of eachexposure frame is acquired, and further a correction parameter of allexposure frames is acquired according to the first brightness data, soas to perform weighting processing on the exposure frame by using thecorrection parameter to obtain an HDR image of corrected Raw data,thereby resolving a problem in the prior art that colors, brightness,and contrast of an image obtained from Raw data are severely distorted.

In another application scenario, the combining a raw data unit arrangedrepeatedly in the exposure frame to obtain first brightness data afterthe combining in the foregoing step 202 includes the following step 2021not shown in the figure.

2021. Acquire a reference brightness value of the exposure frame andcorrect the reference brightness value by using a preset gammacoefficient to obtain the first brightness data.

For example, the acquiring a reference brightness value of the exposureframe may include the following.

1) Acquire an average value of pixels in the raw data unit and use theaverage value as the reference brightness value; for example, in theGRBG Bayer pattern shown in FIG. 1, the reference brightness value maybe (G+R+B+G)/4.

Alternatively,

2) Acquire an average value of G pixels in the raw data unit and use theaverage value as the reference brightness value; for example, in theGRBG Bayer pattern shown in FIG. 1, the reference brightness value maybe (G+G)/2.

The foregoing reference brightness value may be obtained by using atraditional calculation method or by means of simple combination.

That is, in the Bayer pattern shown in FIG. 1, data amount after thecombining is ¼ of the raw data, thereby effectively reducing a dataamount in an image processing process.

It may be known from the foregoing embodiment that a raw data unit isused as a basic unit in an image processing method in this embodiment.First, raw basic units of different exposure frames are preprocessed,such as combination, and first brightness data that is suitable forhuman eyes is output; then a correction parameter required in an HDRimage of corrected raw data is calculated, each piece of raw data in theraw data unit is combined, and color consistency before and after thecombination can be ensured by using a same weighting coefficient andcorrection factor in a same raw data unit; and the brightness data thatis suitable for human eyes is used to calculate a combinationcoefficient, which can implement that an existing brightness postprocessing effect is combined into the output HDR image of the raw data.

In another application scenario, the image processing method is shown inthe following steps S01 to S06. A correction parameter includes aweighting coefficient and a correction factor; each exposure framecorresponds to one weighting coefficient; and all exposure framescorrespond to one correction factor.

Two exposure frames are used as an example in the following.

S01. Collect two exposure frames with different brightness, which are afirst exposure frame and a second exposure frame, in a same scene duringdifferent exposure time.

Brightness of the first exposure frame is less than brightness of thesecond exposure frame, and exposure time of the first exposure frame isless than exposure time of the second exposure frame. R₁ represents thefirst exposure frame, and R₂ represents the second exposure frame.

S02. Combine, for each exposure frame, a raw data unit arrangedrepeatedly in the exposure frame to obtain first brightness data afterthe combining.

First brightness data of the first exposure frame is I′₁, and secondbrightness data of the second exposure frame is I′₂.

Exemplarily, a reference brightness value of the exposure frame isacquired, and the reference brightness value is corrected by using apreset gamma coefficient 1/2.2 in a table lookup manner to obtain thefirst brightness data.

S03. Acquire, for the first exposure frame, weighting coefficient w₁ ofthe first exposure frame according to first brightness data I′₁;acquire, for the second exposure frame, weighting coefficient w₂ of thesecond exposure frame according to second brightness data I′₂.

For example, weighting coefficients w₁ and w₂ are acquired according tothe following formula:

$w_{1}^{\prime} = {{{\exp \left( \frac{- I_{1}^{\prime 2}}{\delta^{2}} \right)}\mspace{14mu} w_{2}^{\prime}} = {\exp \left( \frac{- \left( {255 - I_{2}^{\prime}} \right)^{2}}{\delta^{2}} \right)}}$$w_{1} = {{\frac{w_{1}^{\prime}}{w_{1}^{\prime} + w_{2}^{\prime}}\mspace{14mu} w_{2}} = \frac{w_{2}^{\prime}}{w_{1}^{\prime} + w_{2}^{\prime}}}$

where w′₁ and w′₁ represent intermediate values for calculating theweighting coefficient, and δ represents a Gaussian distributioncoefficient.

S04. Perform brightness processing on the first exposure frame in abrightness histogram equalization manner, so as to obtain secondbrightness data I″₁ after the processing; and perform brightnessprocessing on the second exposure frame in the brightness histogramequalization manner, so as to obtain second brightness data I″₂ afterthe processing.

S05. Acquire, according to weighting coefficients w₁ and w₂, secondbrightness data I″₁ and I″₂, and first brightness data I′₁ and I′₂, acorrection factor a corresponding to all exposure frames:

α=(I″ ₁ *w ₁ +I″ ₂ *w ₂)/(I′ ₁ *w ₁ +I′ ₂ *w ₂).

S06. Perform weighting processing on all exposure frames by usingweighting coefficients w₁ and w₂ and a correction factor a to obtain anHDR image of corrected raw data, which is shown in the followingformula:

R=(R ₁ *w ₁ +R ₂ *w ₂)*α

where R₁ represents the first exposure frame, w₁ represents theweighting coefficient of the first exposure frame, R₂ represents thesecond exposure frame, w₂ represents the weighting coefficient of thesecond exposure frame, α represents the correction factor, and Rrepresents the HDR image of the corrected raw data.

Therefore, an image with undistorted colors, brightness, and contrastcan be obtained by using the foregoing image processing method.

FIG. 4 shows a schematic flowchart of an image processing methodaccording to another embodiment of the present invention. As shown inFIG. 4, steps of the image processing method in this embodiment aredescribed as follows.

401. Collect at least two exposure frames with different brightness in asame scene during different exposure time.

402. Combine, for each exposure frame, a raw data unit arrangedrepeatedly in the exposure frame to obtain first brightness data afterthe combining.

403. Acquire, in a statistical histogram manner, a correspondencebetween first brightness data of a pixel in a bright exposure frame andfirst brightness data of a pixel at a corresponding location in a darkexposure frame so as to obtain a brightness mapping function.

The statistical histogram manner is a histogram obtained with astatistical manner.

It should be noted that herein the method for acquiring the brightnessmapping function is a well-known method for a person skilled in the art.

404. Acquire a correction parameter of all exposure frames according tothe brightness mapping function and all first brightness data andperform weighting processing on all the exposure frames by using thecorrection parameter to obtain an HDR image of corrected raw data.

It may be known from the foregoing embodiment that single-channel rawdata is used in the image processing method to directly acquire the HDRimage of the corrected raw data, thereby reducing the amount ofcalculation in a processing process. In the foregoing image processingmethod, a brightness processing method is used, such as histogramequalization, contrast adjustment, and brightness enhancement of a darkarea. Therefore, color consistency and brightness unity of the raw HDRimage can be maintained.

In another application scenario, the image processing method is shown inthe following steps M01 to M06. A correction parameter includes aweighting coefficient and a correction factor; each exposure framecorresponds to one weighting coefficient; and all exposure framescorrespond to one correction factor.

Two exposure frames are used as an example in the following.

M01. Collect two exposure frames with different brightness, which are afirst exposure frame and a second exposure frame, in a same scene duringdifferent exposure time.

Brightness of the first exposure frame is less than brightness of thesecond exposure frame, and exposure time of the first exposure frame isless than exposure time of the second exposure frame. R₁ represents thefirst exposure frame, a short exposure frame, and a dark exposure frame,and R₂ represents the second exposure frame, a long exposure frame, anda bright exposure frame.

M02. Combine, for each exposure frame, a raw data unit arrangedrepeatedly in the exposure frame to obtain first brightness data afterthe combining.

First brightness data of the first exposure frame is I′₁, and secondbrightness data of the second exposure frame is I′₂.

Exemplarily, M02 may be implemented by performing the followingsubsteps.

M021. Calculate a reference brightness value of the first exposure frameand the second exposure frame.

In step M021, a method for calculating the reference brightness valuemay be a traditional brightness calculation method or may be simplecombination, such as (R+G+G+B)/4. In another embodiment, values of Gpixels may be directly used as the reference brightness value.

The foregoing calculated data amount is ¼ of raw data, therebyeffectively reducing the amount of calculation in a subsequent imageprocessing process.

M022. Perform a gamma correction on the foregoing reference brightnessvalue by using a gamma coefficient to obtain the first brightness data.

It should be noted that the existing gamma correction is the step, whichhas maximum impact on image brightness and contrast, in ISP processing,and also is one important step that transforms the raw data into datasuitable for human eyes. Therefore, step M022 uses a gamma coefficientsame as that in the ISP processing to process reference brightnessvalue.

In an actual application, a value of the gamma coefficient may be 1/2.2.In this embodiment, the gamma correction may be implemented by using atable lookup method. In addition, bit width of the raw data, that is,the raw data of the exposure frame generally is relatively large, suchas 10 bits, 12 bits, or even 14 bits.

To save calculation resources, only the first 8 bits in the raw data areused to perform table lookup, and an output value may also be 8-bitdata.

That is, reference brightness bit width after the gamma correction maynot be the same as Bayer data. Brightness data after the gammacorrection is denoted as I′, that is, the first brightness data of thefirst exposure frame is I′₁, and the second brightness data of thesecond exposure frame is I′₂.

M03. Acquire, in a statistical histogram manner, a correspondencebetween first brightness data of a pixel in a bright exposure frame (thesecond exposure frame) and first brightness data of a pixel at acorresponding location in a dark exposure frame (the first exposureframe) to obtain a brightness mapping function.

For example, brightness mapping function b=f(a), where a represents abrightness value of a pixel in the bright exposure frame, and f(a)represents a brightness value of a pixel at a corresponding location inthe dark exposure frame.

Each group of a and f(a) is referred to as one mapping pair, and adetermined mapping pair is used as the mapping.

That is, if (a, b) is a value of a corresponding lightness mappingfunction f from the long exposure frame (the second exposure frame) tothe short exposure frame (the first exposure frame), b=f(a).

With reference to FIG. 5A, FIG. 5B, and FIG. 5C, the brightness mappingfunction b=f(a) in this embodiment may be easily obtained from astatistical histogram of the first exposure frame and the secondexposure frame.

The following FIG. 5A is the histogram of the first exposure frame andthe second exposure frame. After a simple accumulation, a cumulativehistogram in FIG. 5B may be obtained. After the brightness mapping, abrightness mapping value (the second line is a schematic curve) may beobtained as shown in FIG. 5C, where a great number of overexposurepixels exist in an image of the second exposure frame (an extendedframe) on the horizontal axis.

M04. Acquire a correction parameter of all exposure frames according tothe brightness mapping function and all first brightness data andperform weighting processing on all the exposure frames by using thecorrection parameter to obtain an HDR image of corrected raw data.

The foregoing step M04 may be implemented by performing the followingsubsteps M041 to M044.

M041. Acquire, for the first exposure frame, a weighting coefficient w₁of the first exposure frame according to first brightness data I′₁ andbrightness mapping function b=f(a); and acquire, for the second exposureframe, a weighting coefficient w₂ of the second exposure frame accordingto second brightness data I′₂ and brightness mapping function b=f(a).

For example, weighting coefficients w₁ and w₂ are acquired according tothe following formula:

$w_{1}^{\prime} = {{{\exp \left( \frac{- \left( {255 - a + I_{1}^{\prime}} \right)^{2}}{\delta^{2}} \right)}\mspace{14mu} w_{2}^{\prime}} = {\exp \left( \frac{- \left( {255 + b - I_{1}^{\prime}} \right)^{2}}{\delta^{2}} \right)}}$${w_{1} = {{\frac{w_{1}^{\prime}}{w_{1}^{\prime} + w_{2}^{\prime}}\mspace{14mu} w_{2}} = \frac{w_{2}^{\prime}}{w_{1}^{\prime} + w_{2}^{\prime}}}};$

where w′₁ and w′₂ represent intermediate values for calculating theweighting coefficient, and δ represents a Gaussian distributioncoefficient.

Generally, a dark (I′₁) scene in the long exposure frame (an exposureframe, which has longer exposure time, in the first exposure frame andthe second exposure frame) is more suitable than a dark (I′₂, which isdarker and tends to underexposure) scene in the short exposure frame (anexposure frame, which has shorter exposure time, in the first exposureframe and the second exposure frame).

Similarly, a bright (I′₂) scene in the short exposure is more suitablethan a bright (I′₁, which is brighter and tends to overexposure) scenein the long exposure; therefore, the foregoing formula based on theGaussian distribution is used to calculate the weighting coefficient andimplement normalization.

M042. Perform brightness processing on the first exposure frame in abrightness histogram equalization manner, so as to obtain secondbrightness data I″₁ after the processing; and perform brightnessprocessing on the second exposure frame in the brightness histogramequalization manner, so as to obtain second brightness data I″₂ afterthe processing.

Exemplarily, when the brightness histogram equalization method isadopted, image contrast may be severely changed. For example, a scene inwhich a dynamic range is relatively small has excessively high contrast.Therefore, several key points may be used to control the histogram afterthe mapping. For example, a brightness value corresponding to 25% in thecumulative histogram after the mapping must be between a brightnessvalue corresponding to 25% in a long exposure cumulative histogrambefore the mapping and a brightness value corresponding to 25% in ashort exposure cumulative histogram before the mapping. That is, I′₂_(—) ₂₅≦I′₂₅≦I′₁ _(—) ₂₅.

M043. Acquire, according to the weighting coefficients w₁ and w₂, thesecond brightness data I″₁ and I″₂, and the first brightness data I′₁and I′₂, a correction factor α corresponding to all exposure frames:α=(I″₁*w₁+I″₂*w₂)/(I′₁*w₁+I′₂*w₂).

It should be noted that the exposure time of the first exposure frame isdifferent from the exposure time of the second exposure frame, whichmeans that brightness is different. If simple combination is performedon a corresponding pixel, brightness unity of the final image isdamaged. Therefore, in this embodiment, the correction factor α is setto correct different exposure frames as “one same exposure value”.

M044. Perform weighting processing on all exposure frames by using theweighting coefficients and w₁ and w₂ and the correction factor α toobtain an HDR image of corrected raw data, which is shown in thefollowing formula:

R=(R ₁ *w ₁ +R ₂ *w ₂)*α,

where R₁ represents the first exposure frame, w₁ represents theweighting coefficient of the first exposure frame, R₂ represents thesecond exposure frame, w₂ represents the weighting coefficient of thesecond exposure frame, α represents the correction factor, and Rrepresents the HDR image of the raw data.

One advantage of the foregoing weighting calculation manner depending onthe brightness mapping function is that, when the brightness of the longexposure frame and the short exposure frame is equal to mapping values(a, b), a same combination weight can be obtained, so that quality ofthe output HDR frame is higher.

It may be known from the foregoing embodiment that only single-channelraw data needs to be stored, which reduces the amount of calculation.The brightness processing method may be used directly, such as histogramequalization, contrast adjustment, and brightness enhancement of a darkarea. Color consistency and brightness unity of the HDR image of the rawdata can be maintained.

Particularly, compared with a traditional post processing HDRcombination method, in the present invention, only the single-channeldata needs to be processed, with no need to calibrate a camera responsecurve in advance, thereby effectively reducing storage costs and theamount of calculation of data.

As shown in FIG. 6, FIG. 6-A in FIG. 6 is an HDR image combined in apost processing manner in the prior art, and FIG. 6-B is an HDR image inwhich a post processing combination manner is simply applied to raw dataprocessing (colors and contrast of a gray scale are severely changed inFIG. 6-B); and

FIG. 6-C is an HDR image for which a brightness correction (brightnessunity is damaged, such as the last line in the color chip) is notperformed; and FIG. 6-D is an acquired HDR image after the brightnesscorrection in this embodiment of the present invention (colors andcontrast are closer to a real scene).

In addition, in the foregoing exemplary embodiment, the brightnessmapping function is used in a part of embodiments to calculate theweighting coefficient, and the brightness mapping function is not usedin another part of embodiments to calculate the weighting coefficient.This embodiment sets no limitation thereto, and the brightness mappingfunction may be selected according to an actual need.

Further, in any one of implementation manners listed in this embodimentof the present invention, the weighting coefficient is acquired by usingthe Gaussian distribution method. In another embodiment, such methods asPoisson distribution or Conic may be adopted to acquire the weightingcoefficient, and this embodiment sets on limitation thereto.

According to another aspect of the present invention, an embodiment ofthe present invention further provides an image processing apparatus,which is shown in FIG. 7A and FIG. 7B. The image processing apparatusshown in FIG. 7A and FIG. 7B includes a processor 71 and a memory 72.

The processor 71 is configured to collect at least two exposure frameswith different brightness in a same scene during different exposuretime; combine, for each exposure frame, a raw data unit arrangedrepeatedly in the exposure frame to obtain first brightness data afterthe combining; and acquire a correction parameter of all exposure framesaccording to all first brightness data and perform weighting processingon all the exposure frames by using the correction parameter to obtainan HDR image of corrected raw data; and the memory 72 is configured tostore at least two exposure frames with different brightness that arecollected by the processor.

In an actual application, the processor 71 is configured to acquire, foreach exposure frame, a reference brightness value of the exposure frame,and correct the reference brightness value by using a preset gammacoefficient to obtain the first brightness data; and/or acquire aweighting coefficient of each exposure frame according to all the firstbrightness data; perform brightness processing on each exposure frame toobtain second brightness data after the processing; and acquire,according to the weighting coefficient, the second brightness data, andthe first brightness data, a correction factor corresponding to all theexposure frames, and obtain a correction parameter that includes theweighting coefficient and the correction factor; and/or acquire aweighting coefficient, of each pixel in each exposure frame,corresponding to the exposure frame, and acquire a product obtained bymultiplying the correction factor by each pixel in each exposure frame;summate products corresponding to pixels at corresponding locations inall exposure frames, and use an image formed by all pixels obtainedafter the summating as the HDR image of the corrected raw data.

Optionally, the processor is further configured to acquire, in astatistical histogram manner, a correspondence between first brightnessdata of a pixel in a bright exposure frame and first brightness data ofa pixel at a corresponding location in a dark exposure frame, so as toobtain a brightness mapping function; acquire a weighting coefficient ofeach exposure frame according to the brightness mapping function and allfirst brightness data; perform brightness processing on each exposureframe to obtain second brightness data after the processing; acquire,according to the weighting coefficient, the second brightness data, andthe first brightness data, a correction factor corresponding to all theexposure frames, and obtain a correction parameter that includes theweighting coefficient and the correction factor; further acquire aweighting coefficient, of each pixel in each exposure frame,corresponding to the exposure frame and acquire a product obtained bymultiplying the correction factor by each pixel in each exposure frame;and summate products corresponding to pixels at corresponding locationsin all exposure frames, and use an image formed by all pixels obtainedafter the summating as an HDR image of the corrected raw data.

The processor in this embodiment may perform the image processing methoddescribed in any one of the foregoing embodiments. Details are notdescribed herein again.

It may be known from the foregoing embodiment that according to an imageprocessing apparatus in this embodiment of the present invention, firstbrightness data of each exposure frame is acquired, and further acorrection parameter of all exposure frames is acquired according to thefirst brightness data, so as to perform weighting processing on theexposure frame by using the correction parameter to obtain an HDR imageof corrected Raw data, thereby resolving a problem in the prior art thatcolors, brightness, and contrast of an image obtained from Raw data areseverely distorted.

Persons of ordinary skill in the art may understand that all or a partof the steps of the method embodiments may be implemented by a programinstructing relevant hardware. The program may be stored in a computerreadable storage medium. When the program runs, the steps of the methodembodiments are performed. The foregoing storage medium includes anymedium that can store program code, such as a read-only memory (ROM), arandom-access memory (RAM), a magnetic disk, or an optical disc.

Finally, it should be noted that the foregoing embodiments are merelyintended for describing the technical solutions of the presentinvention, but not for limiting the present invention. Although thepresent invention is described in detail with reference to the foregoingembodiments, persons of ordinary skill in the art should understand thatthey may still make modifications to the technical solutions describedin the foregoing embodiments or make equivalent replacements to some orall technical features thereof, without departing from the scope of thetechnical solutions of the embodiments of the present invention.

What is claimed is:
 1. An image processing method, comprising:collecting at least two exposure frames with different brightness in asame scene during different exposure time; combining, for each exposureframe, a raw data unit arranged repeatedly in the exposure frame toobtain first brightness data after the combining; acquiring a correctionparameter of all exposure frames according to all first brightness data;and performing weighting processing on all the exposure frames by usingthe correction parameter to obtain a high dynamic range (HDR) image ofcorrected raw data.
 2. The method according to claim 1, whereinacquiring the correction parameter of all the exposure frames accordingto all the first brightness data comprises: acquiring a weightingcoefficient of each exposure frame according to all the first brightnessdata; performing brightness processing on each exposure frame to obtainsecond brightness data after the processing; acquiring, according to theweighting coefficient, the second brightness data, and the firstbrightness data, a correction factor corresponding to all the exposureframes; and obtaining a correction parameter that comprises theweighting coefficient and the correction factor.
 3. The method accordingto claim 2, wherein performing the weighting processing on all theexposure frames by using the correction parameter to obtain the HDRimage of the corrected raw data comprises: acquiring a weightingcoefficient, of each pixel in each exposure frame, corresponding to theexposure frame; acquiring a product obtained by multiplying thecorrection factor by each pixel in each exposure frame; summatingproducts corresponding to pixels at corresponding locations in all theexposure frames; and using an image formed by all pixels obtained afterthe summating as the HDR image of the corrected raw data.
 4. The methodaccording to claim 1, wherein combining the raw data unit arrangedrepeatedly in the exposure frame to obtain the first brightness dataafter the combining comprises: acquiring a reference brightness value ofthe exposure frame; and correcting the reference brightness value byusing a preset gamma coefficient to obtain the first brightness data. 5.The method according to claim 4, wherein acquiring the referencebrightness value of the exposure frame comprises: acquiring an averagevalue of pixels in the raw data unit; and using the average value as thereference brightness value.
 6. The method according to claim 4, whereinacquiring the reference brightness value of the exposure framecomprises: acquiring an average value of green (G) pixels in the rawdata unit; and using the average value as the reference brightnessvalue.
 7. The method according to claim 2, wherein two exposure frameswith different brightness, which are a first exposure frame R₁ and asecond exposure frames R₂, are collected in a same scene duringdifferent exposure time, and wherein acquiring the weighting coefficientof each exposure frame according to all the first brightness datacomprises:$w_{1}^{\prime} = {{{\exp \left( \frac{- I_{1}^{\prime 2}}{\delta^{2}} \right)}\mspace{14mu} w_{2}^{\prime}} = {\exp \left( \frac{- \left( {255 - I_{2}^{\prime}} \right)^{2}}{\delta^{2}} \right)}}$${w_{1} = {{\frac{w_{1}^{\prime}}{w_{1}^{\prime} + w_{2}^{\prime}}\mspace{14mu} w_{2}} = \frac{w_{2}^{\prime}}{w_{1}^{\prime} + w_{2}^{\prime}}}};$wherein w′₁ and w′₂ represent intermediate values for calculating theweighting coefficient, w₁ represents a weighting coefficient of thefirst exposure frame, w₂ represents a weighting coefficient of thesecond exposure frame, δ represents a Gaussian distribution coefficient,I′₁ represents first brightness data of the first exposure frame, andI′₂ represents first brightness data of the second exposure frame. 8.The method according to claim 7, wherein acquiring, according to theweighting coefficient, the second brightness data, and the firstbrightness data, the correction factor corresponding to all the exposureframes comprises:α=(I″ ₁ *w ₁ +I″ ₂ *w ₂)/(I′ ₁ *w ₁ +I′ ₂ *w ₂). wherein w₁ representsthe weighting coefficient of the first exposure frame, and w₂ representsthe weighting coefficient of the second exposure frame, and wherein αrepresents the correction factor, I″₁ represents second brightness dataof the first exposure frame, I″₂ represents second brightness data ofthe second exposure frame, I′₁ represents the first brightness data ofthe first exposure frame, and I′₂ represents the first brightness dataof the second exposure frame.
 9. The method according to claim 7,wherein performing the weighting processing on all the exposure framesby using the correction parameter to obtain the HDR image of thecorrected raw data comprises using R=(R₁*w₁+R₂*w₂)*α to obtain an HDRimage R of the corrected raw data when the correction parameter isweighting coefficients w₁ and w₂, and correction factor α, wherein R₁represents the first exposure frame, w₁ represents the weightingcoefficient of the first exposure frame, R₂ represents the secondexposure frame, w₂ represents the weighting coefficient of the secondexposure frame, α represents the correction factor, and R represents theHDR image of the corrected raw data.
 10. The method according to claim1, wherein before acquiring the correction parameter of all the exposureframes according to all the first brightness data, the method furthercomprises acquiring, in a statistical histogram manner, a correspondencebetween first brightness data of a pixel in a bright exposure frame andfirst brightness data of a pixel at a corresponding location in a darkexposure frame to obtain a brightness mapping function, and whereinacquiring the correction parameter of all the exposure frames accordingto all the first brightness data comprises acquiring the correctionparameter of all the exposure frames according to the brightness mappingfunction and all the first brightness data.
 11. The method according toclaim 10, wherein two exposure frames with different brightness, whichare a long exposure frame and a short exposure frame, are collected in asame scene during different exposure time, wherein the brightnessmapping function is b=f(a), wherein a represents a brightness value of apixel in the long exposure frame, and b represents a brightness value ofa pixel which is in the short exposure frame and whose location iscorrespondingly consistent with a location of a, and wherein acquiringthe correction parameter of all the exposure frames according to thebrightness mapping function and all the first brightness data I′₁ andI′₂ comprises: acquiring weighting coefficients w₁ and w₂ of eachexposure frame according to the following formula:$w_{1}^{\prime} = {{{\exp \left( \frac{- \left( {255 - a + I_{1}^{\prime}} \right)^{2}}{\delta^{2}} \right)}\mspace{14mu} w_{2}^{\prime}} = {\exp \left( \frac{- \left( {255 + b - I_{1}^{\prime}} \right)^{2}}{\delta^{2}} \right)}}$${w_{1} = {{\frac{w_{1}^{\prime}}{w_{1}^{\prime} + w_{2}^{\prime}}\mspace{14mu} w_{2}} = \frac{w_{2}^{\prime}}{w_{1}^{\prime} + w_{2}^{\prime}}}};$performing brightness processing on each exposure frame to obtain secondbrightness data after the processing; acquiring, according to theweighting coefficient, the second brightness data, and the firstbrightness data, a correction factor corresponding to all the exposureframes; and obtaining a correction parameter that comprises theweighting coefficient and the correction factor.
 12. The methodaccording to claim 1, wherein the raw data unit is a data unit arrangedin a blue green green red (BGGR) pattern, a data unit arranged in agreen red blue green (GRBG) pattern, or a data unit arranged in a greenred blue white (GRBW) pattern.
 13. An image processing apparatus,comprising: a processor configured to: collect at least two exposureframes with different brightness in a same scene during differentexposure time; combine, for each exposure frame, a raw data unitarranged repeatedly in the exposure frame to obtain first brightnessdata after the combining; acquire a correction parameter of all exposureframes according to all first brightness data; and perform weightingprocessing on all the exposure frames by using the correction parameterto obtain a high dynamic range (HDR) image of corrected raw data; and amemory configured to store at least two exposure frames with differentbrightness that are collected by the processor.
 14. The apparatusaccording to claim 13, wherein the processor is configured to: acquire,for each exposure frame, a reference brightness value of the exposureframe; and correct the reference brightness value by using a presetgamma coefficient to obtain the first brightness data.
 15. The apparatusaccording to claim 13, wherein the processor is configured to: acquire aweighting coefficient of each exposure frame according to all the firstbrightness data; perform brightness processing on each exposure frame toobtain second brightness data after the processing; acquire, accordingto the weighting coefficient, the second brightness data, and the firstbrightness data, a correction factor corresponding to all the exposureframes; and obtain a correction parameter that comprises the weightingcoefficient and the correction factor.
 16. The apparatus according toclaim 13, wherein the processor is configured to: acquire a weightingcoefficient, of each pixel in each exposure frame, corresponding to theexposure frame; acquire a product obtained by multiplying the correctionfactor by each pixel in each exposure frame; summate productscorresponding to pixels at corresponding locations in all exposureframes; and use an image formed by all pixels obtained after thesummating as the HDR image of the corrected raw data.
 17. The apparatusaccording to claim 13, wherein the processor is configured to: acquire,for each exposure frame, a reference brightness value of the exposureframe; correct the reference brightness value by using a preset gammacoefficient to obtain the first brightness data; acquire a weightingcoefficient of each exposure frame according to all the first brightnessdata; perform brightness processing on each exposure frame to obtainsecond brightness data after the processing; acquire, according to theweighting coefficient, the second brightness data, and the firstbrightness data, a correction factor corresponding to all the exposureframes; obtain a correction parameter that comprises the weightingcoefficient and the correction factor; acquire a weighting coefficient,of each pixel in each exposure frame, corresponding to the exposureframe; acquire a product obtained by multiplying the correction factorby each pixel in each exposure frame; summate products corresponding topixels at corresponding locations in all exposure frames; and use animage formed by all pixels obtained after the summating as the HDR imageof the corrected raw data.
 18. The apparatus according to claim 13,wherein the processor is further configured to: acquire, in astatistical histogram manner, a correspondence between first brightnessdata of a pixel in a bright exposure frame and first brightness data ofa pixel at a corresponding location in a dark exposure frame to obtain abrightness mapping function; acquire a weighting coefficient of eachexposure frame according to the brightness mapping function and allfirst brightness data; perform brightness processing on each exposureframe to obtain second brightness data after the processing; acquire,according to the weighting coefficient, the second brightness data, andthe first brightness data, a correction factor corresponding to all theexposure frames; and obtain a correction parameter that comprises theweighting coefficient and the correction factor.